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When  we  first  announced  this  hearing  the  Commerce  Committee 
office  got  dozens  of  telephone  calls  from  people  asking  what  is 
“virtual  reality"  and  why  are  you  holding  a hearing  on  it?  Well,  the 
answer  is  fairly  simple.  Virtual  reality  promises  to  revolutionize 
the  way  we  use  computers.  At  least  that’s  my  opinion.  It  could 
improve  the  way  we  design  new  products,  how  we  teach  our  chil- 
dren, and  how  we  spend  our  free  time.  It  has  the  potential  to 
change  our  lives  in  dozens  of  ways. 

Albert  Gore  opening  statement  at  May  1991  Committee  on  Com- 
merce, Science  and  Transportation  United  States  Senate  Hear- 
ing. 


1.0  Virtual  Environments  - Some  Background 


Virtual  environments  (VE),  alternately  known  as  virtual  reality  (VR),  synthetic  environ- 
ments and  other  names,  are  a collection  of  technologies  which,  offers  the  opportunity  to 
integrate  the  human  into  a computing  system.  The  technologies  which  are  used  to  create 
a virtual  environment  are:  a high  speed  graphics  display  computer,  a head  mounted  dis- 
play, a 3D  position  tracker/input  device,  and  spatial  audio.  When  combined  into  a single 
system  the  computer  application  can  track  a user’s  head  position  and  generate  a display 
that  changes  depending  on  where  the  user  is  looking.  The  3D  input  device  allows  the 
user  to  interact  with  synthetic  objects  and  the  spatial  audio  heightens  a user’s  sense  of 
place. 

For  example  a user  could  interact  with  a virtual  lathe.  The  lathe,  actually  a computer 
graphics  image,  spins  on  the  display  screen  and  the  user  moves  a virtual  cutter  with  a 

Certain  trade  names  and  company  products  are  mentioned  in  the  text  or  identified  in  an  illustration  in  order  to  adequately  specify 
the  experimental  procedure  and  equipment  used.  In  no  case  does  such  identification  imply  recommendation  or  endorsement  by 
the  National  Institute  of  Standards  and  Technology,  nor  does  it  imply  that  the  products  are  necessarily  the  best  available  for  the 
purpose.  This  work  described  was  partially  funded  through  the  NIST  Rapid  Response  Manufacturing  (RRM)  Intramural  Project 
of  the  U.S.  Government  and  is  not  subject  to  copyright. 
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3D  pointing  device  [Sun  92].  Additionally  users  can  sit  in  the  cab  of  a yet-to-be-built 
piece  of  equipment  and  survey  the  visual  obstructions. 

User  interfaces,  the  ways  in  which  we  use  computers,  are  in  general  a bane  to  many 
users.  By  and  large,  computers  are  too  difficult  to  use,  and  a principal  reason  is  poor 
user  interfaces. 

Technical  experts,  for  example,  a machinist  or  shop  foreman,  are  forced  to  become  com- 
puter experts,  generally  against  their  will,  not  because  of  an  interest  in  computers  but 
because  of  the  obtuse  nature  of  the  computer  program  they  need.  Virtual  environments 
offer  the  potential  of  placing  the  user  inside  the  data.  Rather  than  learning  an  obscure 
control  language  in  order  to  operate  a simulation  one  will  be  able  to  simply  reach  out 
and  push  the  data  around.  Of  course,  the  technologies  of  virtual  environments  are  not 
quite  ready  for  prime  time,  however  it  is  also  clear  that  steadily  improving  hardware  and 
software  will  provide  adequate  performance  at  reasonable  prices. 


2.0  Case  Studies 


Even  though  virtual  environment  technologies  are  still  difficult  and  expensive  to  use, 
people  are  doing  real  work.  The  following  case  studies  represent  examples  of  recent 
state  of  the  art  work  that  exemplifies  the  application  of  virtual  environments  in  one  or 
several  aspects  of  manufacturing.  Manufacturing,  in  this  case,  is  taken  to  encompass 
issues  relating  to  maintenance  and  training  as  well  as  the  actual  creation  of  parts  and  the 
assembly  of  systems.  These  examples  are  actual  real  world  systems,  not  simply  specula- 
tive fantasies. 

2.1  Boeing 

The  Research  and  Technology  organization  of  Boeing  Computer  Services  is  actively 
pursuing  two  projects  using  VR  technology.  According  to  David  Mizell,  manager  of 
Virtual  Systems  Research  & Technology,  Boeing  uses  a concept  known  as  Augmented 
Reality. 

Augmented  Reality  is  a term  which  refers  to  the  ability  to  see  through  a computer  gener- 
ated display.  The  generated  images  are  superimposed  on  top  of  reality.  This  is  accom- 
plished by  projecting  a computer  image  onto  a half  silvered  mirror  that  the  user  looks 
through.  This  technique  provides  an  effective  and  intuitive  way  of  “annotating”  reality. 
The  Boeing  team  is  using  a head  set  configured  for  augmented  reality,  which  they  call  a 
HUDset  (Heads-Up,  see-through,  head-mounted  Display). 

The  assembly  of  aircraft  is  a highly  complex  task  that  is  difficult  to  automate.  Many  of 
the  skills  required  demand  dexterity  not  easily  accomplished  by  robots.  In  addition,  air- 
planes consist  of  many  small-lot  size  parts  and  reprogramming  robots  for  these  quanti- 
ties is  an  expensive  prospect.  To  quote  from  Caudell  and  Mizell’s  paper  [Caudell  & 
Mizell  1992]  “Someone  once  said  that  a Boeing  747  is  not  really  an  airplane,  but  five 
million  parts  flying  in  close  formation.”  The  illustration  below  shows  how  a pop-up 
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annotation  points  out  the  location  for  a drilling  operation.  The  annotation  appears  to 
float  in  the  air  on  top  of  the  actual  drilling  site. 


FIGURE  1.  Boeing  HUDSET  (heads  up  display  see  through  head  mounted  display) 

The  particular  task  being  addressed  by  Boeing’s  AR  system  is  the  assembly  of  wire  har- 
nesses. To  quote  again  from  Caudell  and  Mizell’s  paper: 

The  first  of  these  is  the  wiring  formboard,  where  wire  bundles  are 
manufactured  by  laying  individual  wires  across  pegs  on  an  anno- 
tated board.  In  today’s  factory,  the  board  is  annotated  with  a long, 
computer-generated  plot  of  the  wire  bundle  glued  to  sheets  of 
plywood.  The  assembly  worker  must  be  able  to  read  the  paper 
drawing  as  the  bundle  grows.  For  this  demonstration,  a blank  peg 
board  represents  the  formboard.  The  computer  indicates  the 
position  of  the  pegs  for  the  current  wire  bundle  assembly  task  for 
the  user  to  install.  After  this,  the  paths  of  individual  wires  are  indi- 
cated sequentially  on  the  board  as  a series  of  bright  red  lines. 

The  second  application  is  connector  assembly.  After  the  wire 
bundle  is  assembled,  the  ending  leads  are  inserted  into  a mul- 
tipinned  connector.  The  factory  worker  must  consult  a paper 
“map”  of  assignments,  telling  what  wire  goes  into  which  pin.  For 
the  larger  connectors,  as  many  as  fifty  wires  are  mapped  to  pins, 
an  error-prone  task.  In  this  demonstration,  a red  indicating  line 
appears  to  protrude  from  the  connector  at  the  exact  location 
where  the  current  wire  is  to  be  inserted.  A text  window  tells  the 
user  the  number  and  description  of  the  correct  wire.  The  user 
moves  to  the  next  wire  by  again  hitting  the  space-bar  on  the  key- 
board. 

2.2  Caterpillar 

Researchers  at  Caterpillar  Inc.  have  used  VR  to  improve  the  design  process  for  heavy 
equipment  Dave  Stevenson  and  John  Betmer  engineers  with  Caterpillar  in  collabora- 
tion with  the  staff  of  NCSA  (National  Center  for  Supercomputing  Applications)  have 
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put  together  a system  which  allows  them  to  quickly  prototype  wheel  loader  and  backhoe 
loader  designs  [NCSA  1993]. 

In  particular,  the  team  is  able  to  perform  visibility  assessment  of  the  new  design.  Engi- 
neers put  on  a helmet-mounted  display  and  have  a full  360  degrees  of  vision  to  see  how 
the  environment  looks  and  to  evaluate  obstructions.  A Silicon  Graphics  computer  is 
used  to  generate  the  real  time  graphics  display  and  to  simulate  the  operation  of  the 
equipment  The  engineers  can  “operate”  the  equipment  and  evaluate  visual  obstructions 
in  a natural  manner  without  having  to  build  a physical  prototype. 


FIGURE  2.  This  image  from  the  Virtual  Backhoe  project  illustrates  an  "operator”  driving  the  virtual 

equipment  at  the  NCSA  VR  lab. 

The  Caterpillar  team  was  awarded  the  1993  NCSA  Industrial  Challenge  Award  for  VR 
Use.  The  press  release  announcing  the  award  stated: 

This  technology  allows  us  to  dramatically  shorten  the  amount  of 
time  it  takes  to  analyze  a new  design  concept  and  incorporate  it 
into  our  production  process,”  design  Engineer  Dave  Stevenson 
said.  “It  also  represents  a sizeable  cost  savings  because  we 
aren’t  having  to  build  prototype  machines  or  make  last-minute 
design  changes.”  He  said  it  takes  six  to  nine  months  to  build  full- 
scale  models  and  design  changes  using  conventional  design 
methods.  However,  using  the  virtual  reality  approach,  designs 
usually  can  be  evaluated  in  less  than  one  month.  Company  offi- 
cials said  a number  of  design  options  already  been  tested  for  new 
models  of  wheel  loaders  and  backhoe  loaders  that  are  to  be  intro- 
duced by1996,  and  the  company  said  it  eventually  plans  to  allow 
customers  to  “field  test”  new  products  by  putting  on  the  special 
helmet. 
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2.3  Ford 

Within  a development  organization  called  Ford  Alpha  Simultaneous  Engineering  there 
is  an  evaluation  of  VR  for  automotive  assembly  taking  place.  According  to  Jim  Memer, 
[Memer  1992]  manager  of  the  VR  project,  they  are  evaluating  process  installation  feasi- 
bility. The  vehicle  parts  are  represented  in  a CAD  system.  The  CAD  file  is  transferred  to 
the  system  with  the  VR  equipment.  A user  then  manipulates  the  virtual  part  and 
attempts  to  assemble  it  into  the  virtual  vehicle. 

The  equipment  used  for  the  VR  experiments  are  a VPL  eyephone  and  dataglove  running 
on  a Silicon  Graphics  computer.  The  user  puts  all  the  equipment  on  and  attempts  the 
part  insertion.  The  system  checks  for  interference  and  collision  between  the  part  and  the 
vehicle.  The  hope  is  to  use  the  VR  setup  to  evaluate  the  human  ergonomics  of  various 
assembly  operations.  Eventually  they  plan  to  place  some  more  motion  trackers  on  the 
person  to  evaluate  how  much  bending  and  stooping  is  necessary  to  complete  the  assem- 
bly. 

2.4  Matsushita’s  Virtual  Kitchen 

One  of  the  most  widely  publicized  examples  of  VR  used  by  the  public  is  a setup  created 
by  Matsushita  in  Japan.  To  quote  from  Newquist’s  article  [Newquist  1993]: 

The  most  famous  (and  in  danger  of  becoming  something  of  a 
self-caricature)  is  Matsushita’s  Virtual  Kitchen,  a retail  application 
set  up  in  Japan  to  help  people  choose  appliances  and  furnishings 
for  the  relatively  small  kitchen  apartment  spaces  in  Tokyo.  Users 
bring  their  architectural  plans  to  the  Matsushita  store,  and  a vir- 
tual copy  of  their  home  kitchen  is  programmed  into  the  computer 
system.  Buyers  can  then  mix  and  match  appliances,  cabinets, 
colors,  and  sizes  to  see  what  their  complete  kitchen  will  look  like- 
wlthout  ever  installing  a single  Item  In  the  actual  location. 

The  Matsushita  VR  Kitchen  is  significant  because  it  one  of  the  few  examples  of  a VR 
system  set  up  for  public  use  that  is  not  a game  or  in  a research  lab.  The  general  public  is 
invited  to  use  the  configuration. 

2.5  Equipment  Maintenance  - Coiumbia  University 

Some  interesting  work  in  the  use  of  augmented  reality  for  equipment  maintenance  is 
being  done  at  Columbia  University  under  the  direction  of  Steven  Feiner  [Feiner,  MacIn- 
tyre & Seligmann,  92].  The  testbed  application  for  this  project  is  end-user  laser  printer 
maintenance.  The  system  configuration  consists  of  a Reflection  Technology  Private 
Eye,  Logitech  3D  position  trackers  and  a mirror  beam  splitter.  Although  the  scenario 
used  at  Columbia  is  simplistic  their  innovation  lies  in  coupling  a knowledge  based  sys- 
tem with  the  AR  configuration.  Their  system  IBIS  (Intent-Based  Illustration  System)  is 
a rule-based  system  that  designs  illustrations,  according  to  a prioritized  list  of  communi- 
cative goals.  What  this  means  is  that  the  system  plays  an  active  role  in  determining 
where  and  how  information  is  conveyed  to  the  user.  This  reduces  the  amount  of  effort 
needed  to  create  a specific  model  of  the  virtual  world.  The  user  is  shown  only  the  infor- 
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marion  determined  to  be  relevant  to  the  specific  task  at  hand,  and  at  the  appropriate 
time. 

During  operation  of  a paper  filling  operation,  a vector  graphic  display  is  superimposed 
on  top  of  the  real  object  The  toner  cartridge  is  highlighted  with  dotted  lines  and  the 
paper  tray  is  highlighted  with  a textual  callout. 


FIGURE  3.  Experimental  setup  for  augmented  reality  application  at  Columbia  University 

2.6  Maintaining  Equipment  in  Space 

The  Software  Technology  Branch  of  NASA’s  Lyndon  B.  Johnson  Space  Center  is 
exploring  the  use  of  virtual  environment  as  a training  aid.  Maintenance  of  the  Hubble 
Space  Telescope  as  well  as  Space  Station  Freedom  is  difficult.  How  do  you  train  astro- 
nauts to  repair  equipment  in  the  weightlessness  of  space  with  realislic  dynamics  of  the 
objects?  Virtual  environments  which  immerse  the  user  into  a synthetic  space  can  play  an 
important  role  in  training  for  such  maintenance  missions.  [Loftin  93] 

In  an  attempt  to  address  these  issues  two  research  and  development  projects  are  under- 
way. The  objectives  of  the  Hubble  Space  Telescope  Repair  and  Maintenance  are: 

To  develop  a Virtual  Environment  suitable  for  training  astronauts 
in  preparation  for  Hubble  Space  Telescope  Repair  and  Mainte- 
nance missions.  Graphical  models  of  the  Hubble  Space  Tele- 
scope (HST)  and  a large  number  of  digitized  close-out 
photographs  of  the  instrument  are  available.  A virtual  environ- 
ment based  on  a graphical  model  of  the  HST  is  under  develop- 
ment. Where  high  fidelity,  detailed  images  are  needed, 
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appropriate  digitized  images  will  be  displayed.  The  system  is 
designed  to  address  specific  elements  of  future  missions  that  will 
require  location  of  objects  and  interaction  with  handholds,  panels, 
fasteners,  and  orbital  replacement  units. 

A second  demonstration  project,  Training  for  EVA  (Extra  Vehicular  Activity)  Payload 
Interaction  has  as  its  stated  objective: 

To  develop  a proof  of  concept  training  environment  that  provides 
astronauts  with  a simulation  of  satellite  dynamics  that  match 
those  observed  during  the  STS-49  Intellsat  retrieval  mission.  If 
the  proof  of  concept  system  is  judged  to  be  valuable  for  training 
by  training  personnel  and  experienced  astronauts,  a more  com- 
plete model  will  be  developed  and  delivered  as  a training  system 
for  future  payload  retrieval  missions. 

As  was  observed  in  the  1992  rescue  of  the  STS-49  Intellsat  satellite,  the  capture  and 
manipulation  of  large  objects  in  a weightless  state  is  quite  difficult.  It  is  hoped  that  the 
use  of  virtual  environments  will  give  crew  members  an  opportunity  to  develop  and  be 
trained  on  new  procedures.  According  to  Bowen  Loftin,  manager  of  the  program,  the 
initial  demonstration  proof-of-concept  of  a training  environment  will  operate  as  fol- 
lows: 


The  objective  of  this  demonstration  is  to  allow  users  to  experi- 
ence a training  session  in  an  interactive  virtual  environment  by 
exchanging  a faulty  battery  in  the  Hubble  Space  Telescope  (HST) 
with  a new  replacement  battery.  The  user  moves  to  the  HST, 
opens  the  door  to  the  battery  bay,  takes  it  back  to  the  space  shut- 
tle orbiter,  and  places  it  on  top  of  the  hardware  pallet  in  the  pay- 
load  bay.  The  user  than  takes  the  new  battery  from  the  cargo  bay 
hardware  pallet  to  the  HST  and  installs  it  behind  the  battery  bay 
door.  A narrator  will  introduce  a few  details  to  the  user  in  the 
beginning  of  the  session,  and  correct  the  user  when  he/she 
makes  a wrong  move.  The  user  can  grab  the  bay  door  or  the  bat- 
tery by  reaching  out  and  clinching  the  yellow  handle  with  the 
gloved  hand.  Movement  about  the  environment  is  accomplished 
with  the  two  hand-controllers  of  the  chair.  This  chair  emulates  the 
functionality  of  a Manned  Maneuvering  Unit  and  uses  a transla- 
tional control  (left  side)  to  move  in  any  straight  line  by  pulling  or 
pushing  in  the  desired  direction  of  motion.  Also,  a rotational  con- 
trol (right  side)  adjusts  the  user’s  orientation  about  his/her  center 
point.  Changes  are  initiated  in  yaw  by  twisting  the  control  left  or 
right,  in  pitch  by  pulling  it  back  or  pushing  forward,  and  in  roll  by 
tilting  it  left  or  right. 

Benefits  using  a distributed  virtual  environment  are  also  anticipated.  The  Software 
Technology  Branch  is  collaborating  with  the  Marshall  Space  Right  Center  to  allow 
users  at  the  two  centers  to  share  the  same  virtual  environment.  Using  these  techniques, 
joint  training  exercises  could  be  conducted  without  the  usual  collocation  constraints. 


Applying  Virtual  Environments  to  Manufacturing 


7 of  24 


Case  Studies 


2.7  Prototyping  Analysis 

A pioneering  application  of  virtual  reality  was  the  NASA  Windtunnel.  A 3D  worksta- 
tion generates  flow  field  visualizations.  The  user  interacts  with  the  simulation  via  a 
BOOM  display  and  a data  glove.  (A  BOOM  is  a video  display  counterbalanced  on  a 
boom  support  which  the  user  easily  moves  with  their  head.)  The  data  glove  effectively 
acts  as  a source  of  “smoke,”  allowing  the  user  to  observe  local  flow  lines.  The  computa- 
tion of  unsteady  flow  visualization  occurs  in  real  time  and  the  user  can  view  the 
dynamic  scene  from  many  points  of  view  and  scales.  According  to  a description  of  the 
system  [Bryson,  Levit  92]: 

Since  fluid  dynamic  phenomena  occur  over  a large  range  of 
scales  (several  orders  of  magnitude  in  space),  navigation  through 
the  virtual  environment  is  more  difficult  than,  say,  architectural 
walkthrough.  Thus,  In  addition  to  “standard”  viewing  capabilities 
sensitive  to  head  position  and  head  orientation,  our  virtual  envi- 
ronment lets  the  user  rapidly  change  his  or  her  scale,  or  the  scale 
of  the  environment.  Relative  to  the  environment,  a user  can 
shrink  to  become  completely  surrounded  by  some  small  vortex  or 
grow  so  that  the  entire  flow  field  fits  in  one  hand. 


FIGURE  4. 


NASA  Virtual  Wind  Tunnel 
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As  an  extension  to  the  original  virtual  wind  tunnel,  NASA  has  continued  development 
by  examining  feasibility  of  a distributed  virtual  windtunnel  [Bryson,  Gerald-Yamasaki 
92].  In  this  prototype,  several  users  can  interact  with  the  data  being  computed  on  a Con- 
vex C3240  computer  and  visualized  on  a Silicon  Graphic  computer  workstation.  With 
the  resulting  scenario  two  users  can  view  the  same  flow  visualization  and  each  can 
observe  the  flow  lines  (rakes)  injected  by  the  other. 


3.0  Integration  Issues 


Integration  is  perhaps  one  of  the  most  difficult  issues  in  the  domain  of  VE  systems. 
There  are  no  turn-key  solutions  to  manufacturing  problems  and  very  few  turn-key  VE 
systems  at  all.  It  is  difficult,  even  if  you  have  the  money,  to  simply  go  out  and  buy  a 
ready-to-run  VE  system.  One  of  the  only  complete  systems  integrators,  VPL,  has  had 
major  financial  problems.  The  only  other  complete  systems  integrator  is  Division  Incor- 
porated, which  can  provide  a fully  functional  system  on  a variety  of  different  platforms 
[DIVISION  93]. 

One  of  the  significant  barriers  to  the  more  rapid  deployment  of  complete  systems  is  a 
lack  of  standards.  There  are  no  interface  guidelines  for  the  operation  of  a motion  tracker 
with  a graphics  generator  or  a spatial  sound  device.  As  far  as  VE  specifically  there  are 
no  standards  for  the  representation  of  VEs  which  can  be  used  by  a variety  of  VE  sys- 
tems. Many  of  these  issues  have  been  addressed  from  the  CAD  domain  with  standards 
such  as  IGES  [lGES]and  STEP  [ISO];  however,  the  real-time  aspects  of  the  data  are  not 
part  of  those  standards.  Simulation  standards  and  networking  protocols  such  as  used  in 
DARPA’s  SIMNET  project  [Pope  89],  pay  a great  deal  of  attention  to  the  networking 
protocols  necessary  for  real  time  distributed  simulation;  however,  the  detail  for  the  vir- 
tual world  itself  is  somewhat  lacking. 

We  can  view  the  need  for  standards  both  from  a model/world  building  point  of  view  and 
from  an  interface/device  point  of  view.  The  model/world  building  applications  can  draw 
from  the  experiences  of  the  CAD  domain,  and  the  interface/device  issues  can  draw  from 
the  open  systems  domain. 

3.1  Model/World  Building 

Currently  the  development  of  a VE  is  most  easily  accomplished  by  using  a proprietary 
package.  Products  from  companies  such  as  SenseS,  VREAM  and  AutoDesk  allow  users 
to  build  (see  Appendix  B for  company  addresses)  virtual  world;  however,  as  in  many 
CAD  systems  once  a model  is  built  with  a particular  software  package  you  are  stuck  in 
that  world.  Conversion  schemes  to  move  a model  from  one  package  to  another  are  ad- 
hoc  at  best.  Typically  custom  translators  must  be  built  to  convert  the  data. 

As  far  as  manufacturing  data  specifically,  the  integration  of  manufacturing  data  with  a 
virtual  environment  is  no  different  than  the  integration  of  any  other  type  of  data  with  a 
virtual  environment.  From  a conceptual  point  of  view,  however,  manufacturing  data 
does  have  one  advantage  over  purely  abstract  data:  it  is  usually  associated  with  a physi- 
cal object  The  visualization  of  a mechanical  part,  and/or  the  processes  acting  on  that 
part  is  somewhat  easier  to  generate  than  visualizations  of  abstract  concepts  such  as 
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financial  modeling  or  computational  algorithms.  This  is  not  to  minimize  the  difficulties 
of  visualizing  mechanical  processes;  however,  the  physical  part  does  provide  a concep- 
tual metaphor  that  we  can  mentally  grasp. 

3.2  Device  Interfaces 

The  integration  of  hardware  such  as  3D  motion  trackers,  spatial  sound  devices  and  head 
mounted  displays  also  suffers  from  a lack  of  standards.  As  of  this  writing,  there  are  no 
standards  which  would  let  a vendor  write  an  application  that  can  generically  use  one  of 
the  half  dozen  or  so  motion  trackers  available.  The  vendor  would  have  to  either  pur- 
chase a library  (with  the  associated  licensing  costs)  or  use  a VE  development  environ- 
ment that  supports  the  required  devices.  While  not  an  overwhelming  problem  the  lack 
of  interface  standards  is  a barrier  to  the  development  of  more  widespread  applications. 
As  an  example,  there  is  no  standard  for  3D  position  tracking  devices  to  report  their  data. 
This  is  a barrier  to  the  development  of  generic  software  which  can  take  advantage  of 
these  technologies. 

VE  systems  require  a significant  investment  in  hardware.  In  order  to  interface  people  (to 
computers,  not  each  other),  a wide  variety  of  devices  must  be  integrated  into  a system. 
A head-mounted  display  (HMD)  requires  miniature  displays,  a motion  tracker  requires 
some  I/O  to  the  host  computer  as  does  an  audio  system.  There  are  currently  no  standards 
specifically  geared  towards  VE.  Device  I/O  is  a major  problem.  Often  for  example,  a 
sophisticated  3D  motion  tracker  must  be  interfaced  to  the  host  computer  through  a slow 
serial  RS-232  line. 

The  speed  of  low  level  device  I/O  is  an  important  factor  for  VE  systems.  For  example, 
the  typical  scenario  consists  of  a user  wearing  an  HMD  with  a position  tracker.  The 
position  tracker  must  send  information  to  the  computer  which  must  compute  and  render 
new  scenes  based  on  the  position.  If  there  is  a significant  lag  between  the  time  the  user 
moves  his  head  and  the  new  scene  is  generated  and  displayed,  the  entire  immersion 
effect  is  losL  Although  many  factors  contribute  the  overall  performance  of  a system,  I/O 
is  often  a significant  problem. 


4.0  Summary 


Virtual  environment  technology  is  an  evolving  collection  of  technologies.  The  applica- 
tion of  virtual  environments  to  help  solve  manufacturing  problems  gives  manufacturing 
engineers  new  tools  to  solve  complex  problems.  Virtual  environments  offer  the  engineer 
new  ways  to  not  only  visualize  their  problem  spaces  but  to  interact  with  their  environ- 
ment. Many  manufacturing  problems  can  be  readily  visualized  by  images  of  paa  or 
manufacturing  processes.  These  visualizations,  based  on  reality,  can  be  used  as  effec- 
tive conceptual  mechanism  for  the  more  complete  visualizations  embodied  by  iitual 
environments. 

As  demonstrated  by  the  variety  of  case  studies  in  this  paper,  there  are  a wi  ange  of 

applications  for  this  technology.  Two  types  of  applications  in  the  manufacti  ' omain 

which  appear  to  have  great  potential,  particularly  for  rapid  prototyping,  arc  nplified 
by  the  work  at  Caterpillar  and  Ford.  Both  applications  involve  a tight  integ  ...»on  of  the 
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human  into  an  environment.  In  the  case  of  Caterpillar,  the  human  is  placed  into  a syn- 
thetic environment  which  represents  the  prototype  of  a not-yet-built  piece  of  equipment. 
The  user  can  observe  obstructions  and  quickly  see  if  there  are  potential  problems  with 
the  design.  In  the  Ford  application,  the  motion  of  the  human  is  the  key  area  under  study. 
In  this  work  the  question,  “Is  an  assembly  process  humanly  feasible?”  is  being  studied. 
In  both  applications  the  design  process  and  product  development  have  been  sped  up  by 
integrating  the  human  being  directly  into  the  process. 

Although  these  early  applications  are  highly  constrained  they  are  still  useful  first  steps. 
Decisions  made  in  the  design  process  are  the  costliest  to  correct.  Virtual  environment 
techniques  can  improve  the  design  process  by  integrating  human  judgement  at  a far  ear- 
lier stage  of  product  design  then  previously  possible.  Inexpensive  changes  to  the  design 
and  rapid  prototyping  using  these  environments  should  improve  the  quality  and  reduce 
the  development  time  for  new  products. 
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Appendix  A Component  Technologies  for  Virtual  Environments 


A.1  Motion  Trackers  and  3D  Input  Devices 

One  of  the  essential  technologies  needed  to  interact  with  a virtual  environment  is  a 3D 
motion  trackers.  These  devices  which  come  in  a variety  of  forms,  are  used  to  track  the 
position  of  a head,  a hand,  a leg  or  any  object  one  desires  to  use  as  a pointing  device. 
The  literature  often  refers  to  these  devices  as  six  degree  of  freedom  devices  because 
they  can  report  their  position  in  x,  y and  z and  orientation,  roll,  pitch,  and  yaw;  hence 
the  six  degrees  of  freedom. 

Ascension  - A Flock  of  Birds  is  a six  degree  of  freedom  motion  tracking  device  which 
can  be  configured  to  track  up  to  30  receivers  simultaneously.  The  unit  consists  of  one 
transmitter  and  multiple  receivers.  The  flock  may  be  used  in  an  operating  range  of 
approximately  10  feet 


Ascension  Technology  Corp. 
P.O.  Box  527 
Burlington,  VT  05402 
(802)  655-7879 


FIGURES. 


Ascension’s  Flock  of  Bird  (illustrated  with  3 transmitters) 
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Logitech  3D  Mouse  - The  Logitech  3D  Mouse  functions  as  a standard  2D  mouse  or  as  a 
3D  mouse  which  can  report  position,  and  orientation  (roll,  pitch,  and  yaw).  A triangular 
array  of  3 ultrasonic  speakers  sends  signals  to  a triangular  set  of  microphones  on  the 
mouse  that  samples  the  signals  at  up  to  50  times  per  second.  The  mouse  and  triangle 
communicate  along  a line  of  sight  and  both  the  mouse  and  triangle  array  are  connected 
to  a control  unit. 


Logitech  Inc. 

6505  Kaiser  Drive 
Fremont,  CA  94555 
(510)  795-8500 


FIGURE  6. 


Logitech  3-D  mouse 
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Polhemus  - Polhemus  Systems  offers  a 3D  position  tracking  systems  based  on  low-fre- 
quency magnetic  fields.  Their  FastTrak  product  is  a widely  used  solution  for  reliable 
position  tracking. 


Polhemus,  Inc. 

P.O.  Box  560, 

1 Hercules  Drive, 
Colchester,  VT  05446 
(802)  655-3159 


FIGURE  7. 


Polhemus  FASTRAK 


Spatial  Systems,  Inc. 
Middlesex  Technology  Center, 
900  Middlesex  Turnpike, 
Building  8, 

Billerica,  MA  01821 


The  Spaceball  is  a 3-D  control  device  which  looks  like  a billiard  ball  mounted  on  a short 
stick  which  can  sit  next  to  a computer.  Inside  the  ball  is  a 6 degree  of  freedom  sensor 
which  can  be  used  very  effectively  to  position  and  orient  object  or  to  travel  over  some 
data  terrain.  Although  not  typically  thought  of  as  a 3D  tracking  device  the  Spaceball  is 
very  practical  for  the  manipulation  of  objects  in  a conventional  computing  environment. 


Pick  Button 


FIGURES. 


Diagram  of  Spaceball 
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EXOS,  Inc.  has  a high-precision  exoskeleton  device  that  is  worn  like  a glove.  The 
“Dexterous  Hand  Master”  can  track  finger-joint  angles  with  a high  degree  of  resolution. 


EXOS,  Inc. 

8 Blanchard  Road 
Burlington,  MA  01803 
(617)  229-2075 


FIGURE  9.  Dextrous  Hand  Master  (with  3D  position  tracker) 

A.2  Display  Technology 

One  critical  element  required  to  achieve  immersion  into  a virtual  environment  is  the 
ability  to  adjust  a display  while  tracking  the  position  of  the  head.  The  computer  display 
must  remain  in  front  of  your  eyes,  even  when  your  head  moves.  This  is  why  helmets  and 
other  display  devices  that  move  with  you  have  come  into  existence. 
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A.2.1  3D  Devices 

CrystalEyes  is  a LCD  based  3D  display  system.  The  user  wears  a set  of  LCD  glasses 
powered  by  batteries,  so  that  there  are  no  wires.  A transmitter  which  sits  on  top  of  the 
monitor  of  a workstation  synchronizes  the  LCD  shutter  glasses  with  the  computer  dis- 
play, generating  3D  images. 


FIGURE  10.  StereoGraphics  CrystalEyes  3D  display  system 

A.2.2  Booms 

A boom  is  a display  device,  a CRT,  mounted  on  a support  that  counterbalances  the 
weight  of  the  display.  Booms  can  be  configured  either  as  monocular  or  stereo  devices. 
One  of  the  principal  advantages  of  booms  is  that  they  can  support  high  resolution  CRT’s 
not  typical  of  the  Head  Mounted  Displays. 
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Fake  Space  Labs  offers  a number  of  booms  which  incorporate  both  high  resolution 
CRT’s  and  motion  tracking.  They  can  provide  either  black  and  white  or  color  displays. 


Fake  Space  Labs 
935  Hamilton  Ave. 
Menlo  Park,  CA  94025 
(415)  688-1940 


FIGURE  11.  The  Fake  Space  Labs  Boom 

LEEP  Systems  offers  the  CYBERFACE3  system,  a relatively  low  cost  (-$10,000) 
boom  type  display  system. 


LEEP  Systems  Inc., 
241  Crescent  St., 
Waltham,  MA  02154 
(617)  657-1395 


FIGURE  12. 


LEEP  Systems  Cyberface  3 
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A.2.3  Head  Mounted  Display  (HMD) 

n-Vision,  Inc.  offers  some  of  the  highest  quality  head-mounted  displays  available.  Their 
recent  product  is  the  Datavisor  9c  which  can  display  1280x960  stereo  color  images. 
High  resolution  color  CRTs  are  mounted  in  a helmet 


n-Vision  Inc. 

800  Follin  Lane,  Suite  270 
Vienna,  VA  22180 
(703)  242-0030 


RGURE  13.  Datavisor  9C  high  resolution  helmet  mounted  display 

Virtual  Research  offers  the  Eyegen  3 which  can  display  a medium  resolution  color 
images,  mounted  in  a helmet.  The  Eyegen  contains  2 medium  resolution  black  and 
white  CRT  which  are  housed  with  spinning  color  wheels  to  generate  the  color. 


Virtual  Research 
3193  Belick  St.,  Suite  #2 
Santa  Clara,  CA  95054 
(408)  748-8712 


FIGURE  14. 


The  Virtual  Research  Eyegen3 
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A.3  Spatial  Audio 

Audio,  associated  in  three  dimensions  with  objects  can  add  a great  deal  to  the  sensation 
of  immersion-  Devices  that  can  produce  effective  aural  environments  are  available  from 
a number  of  sources,  a sampling  is  listed  below. 

The  Convolvotron  is  a PC  based  board,  and  is  a serious  tool  for  the  study  and  manipula- 
tion of  sound  in  3D. 

Crystal  River  Engineering 
12350  Wards  Ferry  Road 
Groveland,  CA  95321 
(209)  962-6382 

A 3D  audio  device  for  high  end  workstations  such  as  Silicon  Graphics  is  the  SoundCube 
by  VSI. 

Visual  Synthesis,  Inc. 

4126  Addison  Rd. 

Fairfax,  VA  22030 
(703)  352-0258 

A.4  Force  Feedback  Devices 

Force  feedback  devices  give  the  user  the  sensation  feedback  when  their  virtual  pointer 
or  hand  “touches”  a virtual  object.  The  idea  is  to  let  the  user  know,  for  example,  that 
their  hand  has  grasped  an  object.  This  is  an  extremely  immature  technology  when  it 
comes  to  use  in  virtual  environments,  however,  bulky  devices  for  use  in  teleoperation 
and  telerobotics  are  somewhat  more  mature  and  effective  for  particular  remote  applica- 
tions. 

Sarcos  Inc.  manufactures  the  Sarcos  Dextrous  arm,  a force  reflexive  device  in  which 
you  place  your  arm  and  another  robotic  arm  moves  in  the  same  manner.  Although  it  was 
designed  for  use  in  teleoperation  applications  it  may  be  useful  for  virtual  environment 
applications. 

Sarcos  Incorporated, 

261  East  300  South,  Suite  150 
Salt  Lake  City,  UT  84111 
(801)  531-0559 

Another  interesting  device  is  a glove  with  small  air  lines  which  provide  pressure  at 
appropriate  times.  The  device  can  both  sense  and  record  pressure  allowing  objects  to  be 
“felt”  according  to  a previously  sampled  pressure. 

Airmuscle  Ltd. 

12  Orchard  Close 
Cranfield,  Bedford 
United  Kingdom  MK43  OHY 
(44)  234-750-791 
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The  following  describes  a number  of  software  libraries  and  applications  expressly 
designed  for  the  development  of  virtual  environments.  Most  are  C or  C-h-  libraries 
which  can  be  integrated  into  custom  applications  and  all  support  the  various  required 
input  and  output  user  interface  devices. 

B.1  Autodesk  Cyberspace  Developers  Kit 

Autodesk,  developers  of  the  AutoCAD  series  of  products,  has  since  May  1993  offered 
CDK,  the  Cyberspace  Developers  Kit.  CDK  consists  of  C++  class  libraries  designed  to 
facilitate  the  development  of  virtual  environments  on  386  or  486  platforms.  CDK 
includes  support  for  a large  number  of  peripherals  including  head-mounted  displays, 
glove  devices  and  position  trackers.  According  to  Richard  Dym,  general  manager  of 
Autodesk’s  Multimedia  Division,  “We  at  Autodesk  view  Virtual  Reality  as  the  logical 
next  step  in  design  automation,  because  it  adds  interactive  3D  visualization  and  simula- 
tion capabilities  to  the  drafting  and  design  process.”  [Autodesk  1993]  CDK  lists  for 
$2,495.  For  more  information  on  Autodesk  call  1-800-879-4233 

B.2  SENSES  WorldToolKIt 

SENSE8’s  WorldToolKit  2.0  is  a set  of  C functions  that  is  designed  to  run  across  a num- 
ber of  platforms  including  PC’s  and  Silicon  Graphics  workstations.  WorldToolKit  con- 
tains support  for  most  of  the  popular  input  and  output  devices 

SENSE8, 

4000  Bridgeway  Suite  101, 

Sausalito,  CA  94965 
(415)  331-6318 

B.3  University  of  Alberta  Minimal  Reality  (MR)  Toolkit 

The  MR  toolkit  is  a university  developed  subroutine  library  available  with  source  code 
that  supports  the  development  of  virtual  reality  interfaces.  MR  provides  support  for  the 
Polhemus  and  Ascension  motion  trackers,  VPL  DataGlove,  VPL  EyePhone.  The  cur- 
rent version  of  MR  is  callable  from  C and  FORTRAN  on  SGI,  RS6000,  and  DEC  work- 
stations. Unlike  typical  university  projects,  there  is  an  extensive  amount  of 
documentation  the  code  is  actually  documented. 

MR  is  available  at  no  cost  for  research  purposes  by  sending  email  to: 
mr-help<2)  cs  .ualberta.ca 

B.4  VREAM 

The  VREAM  Virtual  Reality  Development  System  from  VREAM,  Inc.  is  a low  cost  PC 
based  system.  VREAM  offers  an  application  called  the  VREAM  3D  World  Editor  and 
the  VREAM  Scripting  Language  as  mechanism  to  build  and  edit  virtual  worlds.  The  3D 
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objects  can  be  controlled  and  manipulated  using  the  VREAM  Runtime  System  and 
there  is  support  for  3D  motion  trackers,  glove,  and  helmet-mounted  displays.  VREAM 
lists  for  $795. 

VREAM  Inc. 

2568  N.  Clark  St.  #250 
Chicago,  IL  60614 
(312)477-0425 
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C.1  Books 

Aukstakalnis,  S.  & Blatner,  D.  (1992).  Silicon  Mirage:  The  Art  and  Science 
of  Virtual  Reality.  Peachpit  Press,  Berkeley,  California. 

An  excellent  book  introducing  all  of  the  relevant  technologies  that 
are  used  to  create  virtual  reality  systems.  Best  of  all  the  hype  so 
often  found  in  the  press  is  left  out.  This  is  a serious  collection  of 
useful  information  and  background  on  VR. 

Blattner,  M.  & Dannenberg,  R.  B.,  eds.  (1992).  Multimedia  Interface 
Design.  ACM  Press/Addison-Wesley,  Reading,  MA 
A collection  of  good  papers  about  multimedia  design  including  sev- 
eral relevant  to  the  individual  technologies  that  go  into  VR  systems. 
These  include  haptic  feedback,  sound  and  novel  metaphors. 

Krueger,  M.  W.  (1991).  Artificial  Reality  II.  Addison- Wesley,  Reading, 
MA. 

Provides  some  good  background  on  the  origins  and  early  “classic” 
works  in  VR.  Tends  to  drift  into  the  more  philosophical  and  idio- 
syncratic views  of  the  author  but  is  still  worthwhile. 

Laurel,  B.  {\99\). Computers  as  Theater.  Addison- Wesley,  Reading,  MA. 
Primarily  a book  about  a new  metaphor  for  thinking  about  user 
interfaces,  that  is  applicable  to  VR.  Includes  a section  specifically 
about  VR  and  discusses,  in  several  places,  how  VR  and  user  inter- 
action are  related. 

C.2  Networks 

CompuServe  contains  a VR  section  which  may  be  accessed  though  the 
COMART  forum  by  typing  GO  COMART  in  the  CompuServe 
information  manager. 

USENET  newsgroup,  sci. virtual-worlds  is  a moderated  newsgroup  con- 
taining discussions  and  announcements  of  the  latest  VR  topics. 

C.3  Periodicals 

CyberEdge  Journal,  #1  Gate  Six  Road,  Suite  G,  Sausalito,  CA  94965 
A bimonthly  newsletter  of  up  to  date  VR  industry  information. 

($  129/year,  six  issues) 
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Sources  of  Information 


IRIS  Universe,  A quarterly  magazine  for  uses  of  Silicon  Graphics  com- 
puter which  usually  has  good  articles  about  VR.  Silicon  Graphics 
2011  North  Shoreline  Blvd.  Mail  Stop  415,  Mountain  View,  CA 
94039-7311  (free  to  qualified  users). 

PRESENCE:  Teleoperators  and  Virtual  Environments,  published  by  The 
MIT  Press,  Cambridge,  MA  02142 
A quarterly  refereed  journal  of  high  quality  articles. 

Virtual  Reality  Report,  published  by  Meckler  Corp.  11  Ferry  Lane  West, 
Westport,  CT  06880  (203)  226-6967.  ($277  / year  - ten  issues). 
Newsletter  of  cuixent  events  and  happening  in  the  VR  world.  Good 
coverage  of  conferences  and  a section  devoted  to  VR  in  Europe. 

SophisTech  Research  (1992).  Virtual  Reality  Sourcebook,  SophisTech 
Research,  6936  Seaborn  St.,  Lakewood  CA,  90713-2832,  (800) 
4VR-SOURCE. 

A comprehensive  reference  of  virtually  everything  and  anything 
having  relevance  to  the  VR  industry.  Is  sold  as  a loose  leaf  collec- 
tion of  pages,  which  unfortunately  does  not  contain  an  index.  $129 
and  a IBM-PC  electronic  version  is  available  for  $95.  Quarterly 
updates  for  $25  ea. 
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